<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  

  
  <title>Chen_Blog</title>
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
  <meta property="og:type" content="website">
<meta property="og:title" content="Chen_Blog">
<meta property="og:url" content="http://yoursite.com/index.html">
<meta property="og:site_name" content="Chen_Blog">
<meta property="og:locale" content="default">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="Chen_Blog">
  
    <link rel="alternate" href="/atom.xml" title="Chen_Blog" type="application/atom+xml">
  
  
    <link rel="icon" href="/favicon.png">
  
  
    <link href="//fonts.googleapis.com/css?family=Source+Code+Pro" rel="stylesheet" type="text/css">
  
  <link rel="stylesheet" href="/css/style.css">
</head>

<body>
  <div id="container">
    <div id="wrap">
      <header id="header">
  <div id="banner"></div>
  <div id="header-outer" class="outer">
    <div id="header-title" class="inner">
      <h1 id="logo-wrap">
        <a href="/" id="logo">Chen_Blog</a>
      </h1>
      
        <h2 id="subtitle-wrap">
          <a href="/" id="subtitle">业精于勤，荒于嬉；行成于思，毁于随。</a>
        </h2>
      
    </div>
    <div id="header-inner" class="inner">
      <nav id="main-nav">
        <a id="main-nav-toggle" class="nav-icon"></a>
        
          <a class="main-nav-link" href="/">Home</a>
        
          <a class="main-nav-link" href="/archives">Archives</a>
        
      </nav>
      <nav id="sub-nav">
        
          <a id="nav-rss-link" class="nav-icon" href="/atom.xml" title="RSS Feed"></a>
        
        <a id="nav-search-btn" class="nav-icon" title="Search"></a>
      </nav>
      <div id="search-form-wrap">
        <form action="//google.com/search" method="get" accept-charset="UTF-8" class="search-form"><input type="search" name="q" class="search-form-input" placeholder="Search"><button type="submit" class="search-form-submit">&#xF002;</button><input type="hidden" name="sitesearch" value="http://yoursite.com"></form>
      </div>
    </div>
  </div>
</header>
      <div class="outer">
        <section id="main">
  
    <article id="stochastic-SGD" class="article article-type-stochastic" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2018/12/18/SGD/" class="article-date">
  <time datetime="2018-12-18T06:11:28.000Z" itemprop="datePublished">2018-12-18</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2018/12/18/SGD/">SGD</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <h1 id="随机梯度下降法"><a href="#随机梯度下降法" class="headerlink" title="随机梯度下降法"></a>随机梯度下降法</h1><p>随机梯度下降（Stochastic Gradient Decent）是对全批量梯度下降法计算效率的改进算法。从本质上来说，预期的随机梯度下降算法的结果和全批量梯度下降法相接近。$$\beta^{i+1} = \beta^{i}-\alpha \triangledown L_{\beta^i}$$<br>此处的loss函数使用的是最小二乘法。<br>$$\triangledown L = (\frac{\partial L}{\partial \beta_0},\frac{\partial L}{\partial \beta_1})=(2(\beta_0+\beta_1<em>x_r-y_r),2</em>x_r(\beta_0+\beta_1*x_r-y_r))$$<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"><span class="keyword">import</span> pandas <span class="keyword">as</span> pd</span><br><span class="line"></span><br><span class="line">train = pd.read_csv(<span class="string">"train.csv"</span>)</span><br><span class="line">test = pd.read_csv(<span class="string">"test.csv"</span>)</span><br><span class="line">submit = pd.read_csv(<span class="string">"sample_submit"</span>)</span><br><span class="line"><span class="comment">#初始设置</span></span><br><span class="line">beta = [<span class="number">1</span>,<span class="number">1</span>]</span><br><span class="line">alpha = <span class="number">0.2</span></span><br><span class="line">tol_L = <span class="number">0.1</span></span><br><span class="line"><span class="comment"># 对x进行归一化</span></span><br><span class="line">max_x = max(train[<span class="string">'id'</span>])</span><br><span class="line">x = train[<span class="string">'id'</span>]/max_x</span><br><span class="line">y = train[<span class="string">'questions'</span>]</span><br><span class="line"></span><br><span class="line"><span class="comment">#定义计算随机梯度的函数</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">compute_grad_SGD</span><span class="params">(beta,x,y)</span>:</span></span><br><span class="line">    grad = [<span class="number">0</span>,<span class="number">0</span>]</span><br><span class="line">    r = np.random.randint(<span class="number">0</span>,len(x))</span><br><span class="line">    grad[<span class="number">0</span>] = <span class="number">2.</span> * np.mean(beta[<span class="number">0</span>] + beta[<span class="number">1</span>] * x[r] - y[r])</span><br><span class="line">    grad[<span class="number">1</span>] = <span class="number">2.</span> * x[r] (beta[<span class="number">0</span>] + beta[<span class="number">1</span>] * x[r] -y[r])</span><br><span class="line">    <span class="keyword">return</span> np.array(grad)</span><br><span class="line"></span><br><span class="line"><span class="comment"># 定义更新beta</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">update_beta</span><span class="params">(beta,alpha,grad)</span>:</span></span><br><span class="line">    new_beta = beta - alpha * grad</span><br><span class="line">    <span class="keyword">return</span> new_beta</span><br><span class="line"></span><br><span class="line"><span class="comment">#定义计算RMSE函数</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">rmse</span><span class="params">(beta,x,y)</span>:</span></span><br><span class="line">    squared_err = (beta_0 + beta_1 * x - y) ** <span class="number">2</span></span><br><span class="line">    res = np.sqrt(np.mean(squared_err))</span><br><span class="line">    <span class="keyword">return</span> res</span><br><span class="line"></span><br><span class="line"><span class="comment">#进行初步的计算</span></span><br><span class="line">np.random.seed(<span class="number">2</span>)</span><br><span class="line">grad = compute_grad_SGD(beta,x,y)</span><br><span class="line">loss = rmse(beta,x,y)</span><br><span class="line">beta = update_beta(beta,alpha,grad)</span><br><span class="line">loss_new = rmse(beta,x,y)</span><br><span class="line"></span><br><span class="line"><span class="comment">#开始迭代</span></span><br><span class="line">i = <span class="number">1</span></span><br><span class="line"><span class="keyword">while</span> np.abs(loss_new - loss) &gt; tol_L:</span><br><span class="line">    beta = update_beta(beta,alpha,grad)</span><br><span class="line">    grad = compute_grad_SGD(beta,x,y)</span><br><span class="line">    <span class="keyword">if</span> i % <span class="number">100</span> == <span class="number">0</span>:</span><br><span class="line">        loss = loss_new</span><br><span class="line">        loss_new = rmse(beta,x,y)</span><br><span class="line">        print(<span class="string">"Round %s Diff RMSE %s "</span>%(i,np.abs(loss_new-loss)))</span><br><span class="line">    i+=<span class="number">1</span></span><br><span class="line">print(<span class="string">'Coef: %s \nIntercept %s'</span>%(beta[<span class="number">1</span>]/max_x, beta[<span class="number">0</span>]))</span><br></pre></td></tr></table></figure></p>

      
    </div>
    <footer class="article-footer">
      <a data-url="http://yoursite.com/2018/12/18/SGD/" data-id="cjptgn1l50001ev15zi6qpbhg" class="article-share-link">Share</a>
      
      
  <ul class="article-tag-list"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Machine-Learning/">Machine Learning</a></li></ul>

    </footer>
  </div>
  
</article>


  
    <article id="post-gradient-descent" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2018/12/18/gradient-descent/" class="article-date">
  <time datetime="2018-12-18T03:52:12.000Z" itemprop="datePublished">2018-12-18</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2018/12/18/gradient-descent/">gradient_descent</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 引用模块</span></span><br><span class="line"><span class="keyword">import</span> pandas <span class="keyword">as</span> pd</span><br><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"></span><br><span class="line"><span class="comment"># 导入数据</span></span><br><span class="line">train = pd.read_csv(<span class="string">'train.csv'</span>)</span><br><span class="line">test = pd.read_csv(<span class="string">'test.csv'</span>)</span><br><span class="line">submit = pd.read_csv(<span class="string">'sample_submit.csv'</span>)</span><br><span class="line"></span><br><span class="line"><span class="comment"># 初始设置</span></span><br><span class="line">beta = [<span class="number">1</span>, <span class="number">1</span>]</span><br><span class="line">alpha = <span class="number">0.2</span></span><br><span class="line">tol_L = <span class="number">0.1</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 对x进行归一化</span></span><br><span class="line">max_x = max(train[<span class="string">'id'</span>])</span><br><span class="line">x = train[<span class="string">'id'</span>] / max_x</span><br><span class="line">y = train[<span class="string">'questions'</span>]</span><br><span class="line"></span><br><span class="line"><span class="comment"># 定义计算梯度的函数</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">compute_grad</span><span class="params">(beta, x, y)</span>:</span></span><br><span class="line">    grad = [<span class="number">0</span>, <span class="number">0</span>]</span><br><span class="line">    grad[<span class="number">0</span>] = <span class="number">2.</span> * np.mean(beta[<span class="number">0</span>] + beta[<span class="number">1</span>] * x - y)</span><br><span class="line">    grad[<span class="number">1</span>] = <span class="number">2.</span> * np.mean(x * (beta[<span class="number">0</span>] + beta[<span class="number">1</span>] * x - y))</span><br><span class="line">    <span class="keyword">return</span> np.array(grad)</span><br><span class="line"></span><br><span class="line"><span class="comment"># 定义更新beta的函数</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">update_beta</span><span class="params">(beta, alpha, grad)</span>:</span></span><br><span class="line">    new_beta = np.array(beta) - alpha * grad</span><br><span class="line">    <span class="keyword">return</span> new_beta</span><br><span class="line"></span><br><span class="line"><span class="comment"># 定义计算RMSE的函数</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">rmse</span><span class="params">(beta, x, y)</span>:</span></span><br><span class="line">    squared_err = (beta[<span class="number">0</span>] + beta[<span class="number">1</span>] * x - y) ** <span class="number">2</span></span><br><span class="line">    res = np.sqrt(np.mean(squared_err))</span><br><span class="line">    <span class="keyword">return</span> res</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment"># 进行初次迭代</span></span><br><span class="line">grad = compute_grad(beta,x,y)</span><br><span class="line">loss = rmse(beta,x,y)</span><br><span class="line">beta = update_beta(beta,alpha,grad)</span><br><span class="line">loss_new = rmse(beta,x,y)</span><br><span class="line"></span><br><span class="line"><span class="comment"># 开始迭代</span></span><br><span class="line">i = <span class="number">1</span></span><br><span class="line"><span class="keyword">while</span> np.abs(loss_new - loss) &gt; tol_L:</span><br><span class="line">    beta = update_beta(beta,alpha,grad)</span><br><span class="line">    grad = compute_grad(beta,x,y)</span><br><span class="line">    loss = loss_new</span><br><span class="line">    loss_new = rmse(beta,x,y)</span><br><span class="line">    i = i +<span class="number">1</span></span><br><span class="line">    print(<span class="string">'Round %s Diff RMSE %s'</span>%(i, abs(loss_new - loss)))</span><br><span class="line">print(<span class="string">'Coef: %s \nIntercept %s'</span>%(beta[<span class="number">1</span>]/max_x, beta[<span class="number">0</span>]))</span><br></pre></td></tr></table></figure>

      
    </div>
    <footer class="article-footer">
      <a data-url="http://yoursite.com/2018/12/18/gradient-descent/" data-id="cjptgn1m10008ev15inbp6sq5" class="article-share-link">Share</a>
      
      
  <ul class="article-tag-list"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Machine-Learning/">Machine Learning</a></li></ul>

    </footer>
  </div>
  
</article>


  
    <article id="post-linear-regression-gluon" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2018/12/06/linear-regression-gluon/" class="article-date">
  <time datetime="2018-12-06T08:04:47.000Z" itemprop="datePublished">2018-12-06</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2018/12/06/linear-regression-gluon/">linear-regression-gluon</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <h1 id="使用Gluon的线性回归"><a href="#使用Gluon的线性回归" class="headerlink" title="使用Gluon的线性回归"></a>使用Gluon的线性回归</h1>
      
    </div>
    <footer class="article-footer">
      <a data-url="http://yoursite.com/2018/12/06/linear-regression-gluon/" data-id="cjptgn1m9000cev15gr5fbce1" class="article-share-link">Share</a>
      
      
    </footer>
  </div>
  
</article>


  
    <article id="post-linear-regression-sources-md" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2018/12/06/linear-regression-sources-md/" class="article-date">
  <time datetime="2018-12-06T06:52:44.000Z" itemprop="datePublished">2018-12-06</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2018/12/06/linear-regression-sources-md/">linear-regression-sources.md</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">from</span> IPython.display <span class="keyword">import</span> set_matplotlib_formats</span><br><span class="line"><span class="keyword">import</span> matplotlib.pyplot <span class="keyword">as</span> plt</span><br><span class="line"><span class="keyword">from</span> mxnet <span class="keyword">import</span> autograd,nd</span><br><span class="line"><span class="keyword">import</span> random</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">set_figsize</span><span class="params">(figsize=<span class="params">(<span class="number">5</span>,<span class="number">4</span>)</span>)</span>:</span></span><br><span class="line">    <span class="string">'''</span></span><br><span class="line"><span class="string">    设置图像</span></span><br><span class="line"><span class="string">    '''</span></span><br><span class="line">    set_matplotlib_formats(<span class="string">'retina'</span>)</span><br><span class="line">    plt.rcParams[<span class="string">'figure.figsize'</span>] = figsize</span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">features_show</span><span class="params">()</span>:</span></span><br><span class="line">    <span class="string">'''</span></span><br><span class="line"><span class="string">    绘制x一个维度与y的散点分布图</span></span><br><span class="line"><span class="string">    '''</span></span><br><span class="line">    set_figsize(figsize=(<span class="number">6</span>,<span class="number">5</span>))</span><br><span class="line">    plt.scatter(features[:,<span class="number">1</span>].asnumpy(),labels.asnumpy())</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">data_iter</span><span class="params">(batch_size,features,labels)</span>:</span></span><br><span class="line">    <span class="string">'''</span></span><br><span class="line"><span class="string">    读取小批量训练数据</span></span><br><span class="line"><span class="string">    '''</span></span><br><span class="line">    num_examples = len(features)</span><br><span class="line">    indx = list(range(num_examples))</span><br><span class="line">    random.shuffle(indx) <span class="comment">#随机打乱，features 的 indx</span></span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">0</span>,num_examples,batch_size):</span><br><span class="line">        j = nd.array(indx[i:min(i+batch_size,num_examples)])</span><br><span class="line">        <span class="comment">#使用take函数根据索引indx找到对应的features</span></span><br><span class="line">        <span class="keyword">yield</span> features.take(j),labels.take(j)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">linreg</span><span class="params">(X,w,b)</span>:</span></span><br><span class="line">    <span class="string">'''</span></span><br><span class="line"><span class="string">    定义线性模型</span></span><br><span class="line"><span class="string">    '''</span></span><br><span class="line">    <span class="keyword">return</span> nd.dot(X,w)+b</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">squared_loss</span><span class="params">(y_hat,y)</span>:</span></span><br><span class="line">    <span class="string">'''</span></span><br><span class="line"><span class="string">    定义损失函数</span></span><br><span class="line"><span class="string">    '''</span></span><br><span class="line">    <span class="keyword">return</span> ((y_hat - y.reshape(y_hat.shape)) ** <span class="number">2</span>) / <span class="number">2</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">sgd</span><span class="params">(params,lr,batch_size)</span>:</span></span><br><span class="line">    <span class="keyword">for</span> param <span class="keyword">in</span> params:</span><br><span class="line">        param[:] = param - lr * param.grad/batch_size</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">main</span><span class="params">()</span>:</span></span><br><span class="line">    lr = <span class="number">0.01</span></span><br><span class="line">    num_epochs = <span class="number">4</span></span><br><span class="line">    batch_size = <span class="number">10</span></span><br><span class="line">    num_inputs = <span class="number">2</span></span><br><span class="line">    num_examples = <span class="number">1000</span></span><br><span class="line">    true_w = [<span class="number">2</span>,<span class="number">-3.4</span>]</span><br><span class="line">    true_b = <span class="number">4.2</span></span><br><span class="line">    features = nd.random.normal(scale = <span class="number">1</span>,shape=(num_examples,num_inputs))</span><br><span class="line">    labels = true_w[<span class="number">0</span>] * features[i][<span class="number">0</span>] + true_w[<span class="number">1</span>] * features[i][<span class="number">1</span>] + true_b</span><br><span class="line">    labels += nd.random.normal(scale=<span class="number">0.1</span>,shape=(labels.shape))</span><br><span class="line">    <span class="comment"># 预测模型</span></span><br><span class="line">    w = nd.random.normal(scale=<span class="number">0.01</span>,shape=(num_inputs,<span class="number">1</span>))</span><br><span class="line">    b = nd.random.normal(scale=<span class="number">1</span>,shape=(<span class="number">1</span>,))</span><br><span class="line">    params = [w,b]</span><br><span class="line">    <span class="comment">#训练的时候需要对这些参数求梯度，来跌打他们的值，所以需要创建他们的梯度</span></span><br><span class="line">    <span class="keyword">for</span> param <span class="keyword">in</span> params:</span><br><span class="line">        param.attach_grad()</span><br><span class="line"></span><br><span class="line">    net = linreg</span><br><span class="line">    loss = squared_loss</span><br><span class="line">    set_figsize(figsize=(<span class="number">7</span>,<span class="number">6</span>))</span><br><span class="line">    features_show()</span><br><span class="line">    <span class="keyword">for</span> epoch <span class="keyword">in</span> range(<span class="number">1</span>,num_epochs+<span class="number">1</span>):</span><br><span class="line">        <span class="keyword">for</span> X,y <span class="keyword">in</span> data_iter(batch_size,features,labels):</span><br><span class="line">            <span class="keyword">with</span> autograd.record():</span><br><span class="line">                l = loss(net(X,w,b),y)</span><br><span class="line">            l.backward()</span><br><span class="line">            sgd([w,b],lr,batch_size)</span><br><span class="line">        print(<span class="string">"epoch&#123;0&#125;,loss&#123;1&#125;"</span>.format(epoch,loss(net(X,w,b),y).mean().asnumpy()))</span><br><span class="line">    print(true_w,w)</span><br><span class="line">    print(true_b,b)</span><br><span class="line"><span class="comment">#     print("true_w[0]:&#123;0&#125;,true_w[1]:&#123;1&#125;.w[0]:&#123;2&#125;,w[1]:&#123;3&#125;,true_b:&#123;4&#125;,b:&#123;5&#125;".\</span></span><br><span class="line"><span class="comment">#           format(true_w[0],true_w[1],w[0],w[1],true_b,b))</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> __name__ == <span class="string">"__main__"</span>:</span><br><span class="line">    main()</span><br></pre></td></tr></table></figure>

      
    </div>
    <footer class="article-footer">
      <a data-url="http://yoursite.com/2018/12/06/linear-regression-sources-md/" data-id="cjptgn1mc000eev15mchbsvy7" class="article-share-link">Share</a>
      
      
    </footer>
  </div>
  
</article>


  
    <article id="linear-regression" class="article article-type-linear" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2018/12/05/regression/" class="article-date">
  <time datetime="2018-12-05T14:14:35.000Z" itemprop="datePublished">2018-12-05</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2018/12/05/regression/">regression</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <h1 id="线性回归-linear-regression"><a href="#线性回归-linear-regression" class="headerlink" title="线性回归 linear regression"></a>线性回归 linear regression</h1><p>设训练数据样本集为1000，输入个数为2，给定随机产生的批量样本特征$\boldsymbol{X} \in \mathbb{R}^{1000\times2}$我们使用真实的线性回归模型，真实的权重$\boldsymbol{w}=[2,-3.4]^\top$和偏差$b = 4.2$添加一个噪音项，$\epsilon$生成目标变量$\boldsymbol{Y}$<br>$$\boldsymbol{Y}=\boldsymbol{w} \boldsymbol{X}+b +\epsilon$$<br>此示例，其中噪音项$\epsilon\sim N(0,0.001)$正态分布。生成数据集如下：<br>$$y[i]=w[0] <em> X[i][0] + w[1] </em> X[i][1] + b + \epsilon $$</p>
<p>损失函数：</p>
<p>$$squared-loss = argmin\frac{1}{2}\sum (\hat y -y)^2$$<br>$$\frac{\partial E_{w,b}}{\partial w} = 2 (w \sum^m_{i=1}x^2_i - \sum^m_{i =1}(y_i-b)x_i )$$</p>
<p>$$\frac{\partial E_{w,b}}{\partial b} = 2 (mb - \sum^m_{i =1}(y_i-wx_i) )$$</p>
<p>$$ \frac{\partial E_{w,b}}{\partial w}=0 $$<br>$$ \frac{\partial E_{w,b}}{\partial b}=0 $$<br>1.<br>$$b = \frac{1}{m}\sum^m_{i=1}(y_i - wx_i)$$<br>2.<br>$$ \frac{\partial E_{w,b}}{\partial w}=0 $$<br>3.<br>$$w\sum^m_{i=1}x^2_i-\sum^m_{i =1}(y_ix_i-bx_i)=0$$<br>4.<br>$$w\sum^m_{i=1}x^2_i-\sum^m_{i =1}y_ix_i+\sum^m_{i =1}bx_i=0$$<br>5.<br>$$w\sum^m_{i=1}x^2_i-\sum^m_{i =1}y_ix_i+b\sum^m_{i =1}x_i=0$$<br>6.<br>$$w\sum^m_{i=1}x^2_i-\sum^m_{i =1}y_ix_i+(\frac{1}{m}\sum^m_{i=1}(y_i - wx_i))\sum^m_{i =1}x_i=0$$<br>7.<br>$$w\sum^m_{i=1}x^2_i-\sum^m_{i =1}y_ix_i+\frac{1}{m}\sum^m_{i=1}y_i\sum^m_{i =1}x_i -\frac{1}{m}\sum^m_{i=1} wx_i\sum^m_{i =1}x_i=0$$<br>8.<br>$$w(\sum^m_{i=1}x^2_i-\frac{1}{m}(\sum^m_{i =1}x_i)^2)=\sum^m_{i =1}y_ix_i-\sum^m_{i=1}y_i\bar x$$<br>9.<br>$$w = \frac{\sum^m_{i =1}y_i(x_i-\bar x)}{\sum^m_{i=1}x^2_i-\frac{1}{m}(\sum^m_{i =1}x_i)^2}$$</p>
<p>$$w = \frac{\sum^m_{i=1}y_i(x_i-\bar x)}{\sum^m_{i=1}x^2_i-\frac{1}{m}(\sum^m_{i=1}x_i)^2}$$<br>$$b = \frac{1}{m}\sum^m_{i=1}(y_i - wx_i)$$</p>
<p>arg    是变元（即自变量argument）的英文缩写。<br>arg min 就是使后面这个式子达到最小值时的变量的取值<br>arg max 就是使后面这个式子达到最大值时的变量的取值</p>
<p>例如 函数F(x,y):</p>
<p>arg  min F(x,y)就是指当F(x,y)取得最小值时，变量x,y的取值</p>
<p>arg  max F(x,y)就是指当F(x,y)取得最大值时，变量x,y的取值</p>
<p>通过梯度求导 得到适合的 w 和 b</p>
<p>为了便于讨论，我们把$w$和$b$写成向量的形式$\hat w = (w,b)$，相应的，把数据集D表示为一个$m\times (d+1)$大小的矩阵$X$，则有$$\hat w^* =argmin_{\hat w}(y - X \hat w)^T(y - X \hat w)$$<br>令$E_{\hat w}=(y - X \hat w)^T(y - X \hat w)$ 对$\hat w$进行求导<br>1.<br>$$<br>\begin{aligned}<br>&amp; \frac{\partial E \hat w}{\partial w} =-2X^T (y-X\hat w) \<br>&amp;<br>\end{aligned}<br>$$<br>令$\frac{\partial E_{\hat w} }{\partial \hat w}=0$,可得到$\hat w$的最优解$$\hat w = (X^TX)^{-1}X^Ty$$<br>令$\hat{x}_i=(x_i,1)$,则多元线性回归模型为<br>$$<br>f(\hat{x}_i)=\hat{x_i}^T(X^TX)^{-1}X^Ty<br>$$<br>另外一种推导方式<br>同样地，我们使用最小二乘法对w和b进行估计，令均方误差的求导等于0，计算过程如下：<br>   令$E_{\hat{w}}=(y-x\hat{w})^T(y-x\hat{w})$  </p>
<p>   $$<br>   \begin{aligned}<br>   E_{\hat{w}}&amp;=(y^T-(X\hat{w})^T))(y-x\hat{w})\<br>   &amp;=(y^T-X^T\hat{w}^T)(y-x\hat{w})\<br>   &amp;=(X^T\hat{w}^T-y^T)(x\hat{w}-y)\<br>   &amp;=\hat{w}^TX^TX\hat{w}-y^TX\hat{w}-\hat{w}^TX^Ty+y^Ty\<br>   &amp;=\hat{w}^TX^TX\hat{w}-y^TX\hat{w}-y^TX\hat{w}+y^Ty\<br>   &amp;=\hat{w}^TX^TX\hat{w}-2y^TX\hat{w}+y^Ty<br>   \end{aligned}<br>   $$<br>   对$\hat{w}$求导<br>   $$<br>   \begin{aligned}<br>   \frac{\partial{E_{\hat{w}}}}{\partial{\hat{w}}}&amp;=(X^TX+(X^TX)^T)\hat{w}-2y^TX,\frac{\partial{X^TAX}}{\partial{X}}=(A+A^T)X\<br>   &amp;=2X^TX\hat{w}-2X^Ty<br>   \end{aligned}<br>   $$<br>   令上式=0，可得<br>   $$<br>   \hat{w}^*=(X^TX)^{-1}X^Ty<br>   $$<br>   其中$(X^TX)^{-1}$是$(X^TX)$的逆矩阵，$(X^TX)$是满秩矩阵或非奇异矩阵<br>   令$\hat{x}_i=(x_i,1)$,则多元线性回归模型为<br>   $$<br>   f(\hat{x}_i)=\hat{x_i}^T(X^TX)^{-1}X^Ty<br>   $$</p>
<h2 id="对数几率回归"><a href="#对数几率回归" class="headerlink" title="对数几率回归"></a>对数几率回归</h2><p>使用线性模型进行回归学习。<br>“单位阶跃函数”<br>$$ y = \left {<br>    \begin{aligned}<br>    0 , &amp; z&lt;0;\<br>    0.5, &amp; z=0;\<br>    1, &amp; z&gt;0;<br>    \end{aligned}<br>\right.<br>$$<br>由于单位阶跃函数不是连续的，所以不能直接求导，所以我们希望能在找到可以替代的连续可微的单调函数，如下：<br>$$ y = \frac{1}{1+e^{-z}}$$<br>对数几率函数是一种”Sigmoid函数”<br>$$\begin{aligned} &amp;&amp; y = \frac{1}{1+e^{-(\boldsymbol W^Tx+b)}}\<br>&amp;&amp; y(1+e^{-(\boldsymbol W^Tx+b)})=1 \<br>&amp;&amp; e^{-(\boldsymbol W^Tx+b)} = \frac{1}{y} -1\<br>&amp;&amp; -(\boldsymbol W^Tx+b) = ln\frac{1-y}{y} \<br>&amp;&amp; -(\boldsymbol W^Tx+b )= ln(\frac{y}{1-y})^{-1} \<br>&amp;&amp; -(\boldsymbol W^Tx+b) = ln\frac{1-y}{y} \<br>&amp;&amp; -(\boldsymbol W^Tx+b )= -ln(\frac{y}{1-y}) \<br>&amp;&amp; (\boldsymbol W^Tx+b )= ln(\frac{y}{1-y})<br>\end{aligned}<br>$$<br> 如果将y视为样本x的正例的可能性，则1-y是其反例可能性，两者的比率为$\frac{y}{1-y}$对数几率回归又称为logit regression 虽然是叫做回归但是解决的是分类问题。</p>

      
    </div>
    <footer class="article-footer">
      <a data-url="http://yoursite.com/2018/12/05/regression/" data-id="cjptgn1mb000dev154uqvwm0s" class="article-share-link">Share</a>
      
      
  <ul class="article-tag-list"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/linear-regression/">linear regression</a></li></ul>

    </footer>
  </div>
  
</article>


  
    <article id="post-PseudoBaseStation" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2018/08/21/PseudoBaseStation/" class="article-date">
  <time datetime="2018-08-21T07:54:47.000Z" itemprop="datePublished">2018-08-21</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2018/08/21/PseudoBaseStation/">PseudoBaseStation</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <p>[TOC]</p>
<p>IMSI是国际移动用户识别码 ，是区别移动用户的标志，储存在SIM卡中，可用于区别移动用户的有效信息</p>
<p>手机终端每隔一段时间（例如5秒）会扫描附近运营商的信号，选取信号强的基站进行连接</p>
<p>终端连接基站的时候如果有位置区LAC（2G）或跟踪区TAC(4G)变化，那么终端就会发起位置更新请求</p>
<p>由于系统必须要识别终端是否合法，所以终端会应要求，然后上传IMSI、IMEI等信息</p>
<p>伪基站就是利用这一动作，伪装成正常基站，诱导终端连接到伪基站上，通过位置更新动作，抓取用户IMSI，并伪装成任意号码对所有抓取到的IMSI群发短信，达到广告、钓鱼、诈骗的目的</p>
<h1 id="伪基站识别"><a href="#伪基站识别" class="headerlink" title="伪基站识别"></a>伪基站识别</h1><p>伪基站通过信号强度将手机吸入网络，手机发现进入一个新的位置区，向伪基站发起位置变更请求，伪基站不可能像正常基站一样通过VLR、MSC、HLR查询IMSI，但根据正常流程可以向MS要求上报IMSI，于是手机上报IMSI等信息被伪基站获取</p>
<p>手机被踢出伪基站网络后，会带上变更前的位置区信息访问正常网络，这个时候就可以根据其更新前的位置区信息来辨别伪基站</p>
<h2 id="通过LAC识别2G伪基站"><a href="#通过LAC识别2G伪基站" class="headerlink" title="通过LAC识别2G伪基站"></a>通过LAC识别2G伪基站</h2><p>用户被踢出伪基站的时候，在信令数据中会生成位置变更的数据，包含变更前和变更后的LAC和CI</p>
<p>变更前基站的LAC、CI会是异常值，例如65534、0、或异地基站，或无法查询到的基站，统计变更后基站下异常基站数据出现次数，分析得出伪基站出没小区</p>
<h2 id="通过TAC识别4G伪基站"><a href="#通过TAC识别4G伪基站" class="headerlink" title="通过TAC识别4G伪基站"></a>通过TAC识别4G伪基站</h2><p>4G基站和2G基站识别位置区域的方式不同，通过TAC、CI、PCI定位一个小区，但都是通过位置更新获取用户信息，其信令包含位置变更前后的TAC、PCI</p>
<p>统计变更前后异常TAC、PCI出现次数，分析得出伪基站出没小区</p>
<h2 id="通过LTE网络的ANR功能识别4G伪基站"><a href="#通过LTE网络的ANR功能识别4G伪基站" class="headerlink" title="通过LTE网络的ANR功能识别4G伪基站"></a>通过LTE网络的ANR功能识别4G伪基站</h2><p>准备全网邻区配置表</p>
<p>准备一定时间内服务小区和邻区关系</p>
<p>如果小区下有上报CGI，那么该小区下有UE开启ANR功能</p>
<p>统计小区内邻区一定时间内漏配邻区上报数量</p>
<p>统计小区内邻区一定时间内信号强度大的数量</p>
<p>将漏配邻区数量和信号强度异常的数量综合分析，得出伪基站出没小区</p>
<h2 id="通过基站用户数变动情况识别伪基站"><a href="#通过基站用户数变动情况识别伪基站" class="headerlink" title="通过基站用户数变动情况识别伪基站"></a>通过基站用户数变动情况识别伪基站</h2><p>伪基站覆盖范围能够达到几百米，通常出现在人数较多的区域，工作时会通过一些手段屏蔽附近网络，比如用超出运营商基站的信号强度的信号，用户会断开与正常基站的连接，10s中左右会重新连回到正常基站，这段时间基站下连接频次必然会出现波动</p>
<p>所以可统计一定周期内基站下号码出入网络频数，根据频次波动情况可分析基站下可能存在伪基站</p>
<h2 id="伪基站轨迹"><a href="#伪基站轨迹" class="headerlink" title="伪基站轨迹"></a>伪基站轨迹</h2><p>当伪基站被识别后，可以按伪基站基站位置随时间的变化来推断伪基站轨迹</p>
<p>也可以通过长时间的伪基站识别，区分仿真基站、公安伪基站等</p>
<h1 id="伪基站工作原理"><a href="#伪基站工作原理" class="headerlink" title="伪基站工作原理"></a>伪基站工作原理</h1><h2 id="手机吸入"><a href="#手机吸入" class="headerlink" title="手机吸入"></a>手机吸入</h2><ul>
<li>工程手机获取邻近小区基站的广播频率，并测量场强</li>
<li>选择广播信号最弱的小区频率，发射伪装后的广播，广播信号强度相对较强，位置区和当前位置区不同</li>
<li>手机扫描信号后接入伪基站</li>
<li>手机发现位置区变更，发起位置更新</li>
<li>伪基站发出识别请求</li>
<li>手机反馈IMSI、IMEI等，伪基站成功获取用户信息</li>
<li>伪基站根据IMSI判断是否已发送短信，若未发，则设置任意主叫号码，于独立专用信道发送短信</li>
</ul>
<h2 id="手机踢出"><a href="#手机踢出" class="headerlink" title="手机踢出"></a>手机踢出</h2><p>手机小区重选，接入正常基站</p>
<ul>
<li>正常位置更新<ul>
<li>伪基站变更位置区并广播</li>
<li>手机发现位置区变更，发起位置更新</li>
<li>伪基站判断是否已发短信，已发则拒绝位置更新</li>
<li>小区重选，接入正常基站</li>
</ul>
</li>
<li>周期性位置更新<ul>
<li>伪基站设置计时器</li>
<li>用户按照计时器计时</li>
<li>逾时触发位置更新</li>
<li>伪基站拒绝位置更新</li>
<li>小区重选，接入正常基站</li>
</ul>
</li>
<li>手机终端发现无法使用服务</li>
</ul>
<h1 id="手机位置更新"><a href="#手机位置更新" class="headerlink" title="手机位置更新"></a>手机位置更新</h1><h2 id="正常位置更新"><a href="#正常位置更新" class="headerlink" title="正常位置更新"></a>正常位置更新</h2><p>正常位置更新的前提条件</p>
<ul>
<li>VLR中MS的状态未知</li>
<li>MS从一个LAI小区重选至另外一个LAI小区</li>
</ul>
<h3 id="VLR内部位置更新"><a href="#VLR内部位置更新" class="headerlink" title="VLR内部位置更新"></a>VLR内部位置更新</h3><p>VLR内部位置更新不需要提供IMSI号码，在VLR中进行，不需要通知HLR.</p>
<ul>
<li>在初始化过程中，MS向网络发送位置更新请求，并携带MS的TMSI号码及LAI号码，并标注为正常位置更新。</li>
<li>MSC收到MS发送的位置更新请求后，将向VLR发送位置区更新消息。</li>
<li>VLR收到位置更新消息后进行位置更新处理，VLR将更新MS位置消息并存储新的LAI号码，并根据需要给MS分配一个新的TMSI号码（此时进入TMSI再分配程序）。</li>
</ul>
<blockquote>
<p>注意：此时TMSI再分配命令也可以不携带新的TMSI号码，MS将使用以前的TMSI号码。</p>
</blockquote>
<ul>
<li>当收到MS发送给网络的TMSI再分配完成消息后，VLR将向MSC发送位置区更新确认消息。</li>
<li>MSC收到该消息后则向MS发送位置更新接受消息，之后释放信道，完成位置更新。</li>
</ul>
<h3 id="越VLR的位置更新"><a href="#越VLR的位置更新" class="headerlink" title="越VLR的位置更新"></a>越VLR的位置更新</h3><ul>
<li>在初始化过程中，MS向网络发送位置更新请求，并携带MS的TMSI号码及LAI号码，并标注为正常位置更新。</li>
<li>MSC收到MS发送的位置更新请求后，将向VLR发送位置区更新消息。</li>
<li>当VLR从MSC收到位置更新消息中的TMSI未知，根据旧TMSI和LAI号算出PVLR地址</li>
<li>向PVLR启动一个请求IMSI和鉴权参数的发参数指示。</li>
<li>PVLR将回发该MS的IMSI和鉴权参数。</li>
<li>当VLR由于种种原因无法获得IMSI号码，则向MS发出识别请求，请求MS提供IMSI。</li>
<li>VLR得到IMSI后，将向MS所属的HLR发出位置更新消息，此消息中包括MS的标识及相关信息，以便HLR查询数据及建立路径。</li>
<li>VLR将对MS进行鉴权加密，并根据需要给MS分配一个新的TMSI号码（此时进入TMSI再分配程序）。</li>
</ul>
<blockquote>
<p>注意：此时TMSI再分配命令也可以不携带新的TMSI号码，MS将使用以前的TMSI号码。</p>
</blockquote>
<ul>
<li>HLR收到更新消息后，并且MS在新的VLR有正常业务权限，则HLR存储当前的VLR号码，并向PVLR发出删除位置消息。</li>
<li>PVLR收到删除位置消息后，将删除该MS的所有信息，并向HLR发送删除位置确认消息。</li>
<li>当完成鉴权加密TMSI再分配后，HLR将发起插入用户数据消息，为VLR提供所需用户信息，包括鉴权参数等。</li>
<li>VLR收到所需信息后，将向HLR发送插入用户数据响应消息。</li>
<li>当HLR收到VLR插入用户数据响应后，则向VLR发出更新确认消息</li>
<li>之后VLR将向MSC发送位置区更新确认消息。</li>
<li>MSC收到该消息后则向MS发送位置更新接受消息，之后释放信道，完成位置更新。</li>
</ul>
<h3 id="IMSI更新"><a href="#IMSI更新" class="headerlink" title="IMSI更新"></a>IMSI更新</h3><p>当用户识别为IMSI时：</p>
<ul>
<li><p>当VLR从MSC收到位置更新消息中的IMSI未知，则将发起HLR更新</p>
</li>
<li><p>当收到IMSI是已知的， VLR则检查从MSC接收的们前一个位置区标识（LAI）是否属于此VLR，如果不属于则发起HLR更新。</p>
</li>
</ul>
<blockquote>
<p>注意：当以上两种情况发生时，均要进行鉴权检查</p>
</blockquote>
<h2 id="周期性位置更新"><a href="#周期性位置更新" class="headerlink" title="周期性位置更新"></a>周期性位置更新</h2><p>周期位置更新发生在当网络在特定的时间内没有收到来自MS任何信息。比如在某些特定条件下由于无线链路质量很差，网络无法接收MS的正确消息，而此时MS还处于开机状态并接收网络发来的消息，在这种情况下网络无法知道MS所处的状态。为了解决这一问题，系统采取了强制登记措施。如系统要求移动用户在一特定时间内，例如一个小时，登记一次。这种位置登记过程就叫做周期位置更新。</p>
<blockquote>
<p>注意：</p>
<p>当T3212逾时后，MS启动周期性位置更新，进入位置更新程序。</p>
<p>周期性位置更新信令流程与正式常位置更信令流程是一致的。</p>
</blockquote>
<p><strong>网络失去与MS的联系原因：</strong></p>
<ul>
<li><p>MS开机移动至网络盲区，网络仍会认为IMSI附着（用户开机）；</p>
</li>
<li><p>MS IMSI分离（关机）时，无线路径上行链路故障，网络不能正确译码，网络仍会认为IMSI附着（用户开机）；</p>
</li>
<li><p>MS突然掉电，网络仍会认为IMSI附着（用户开机）；</p>
</li>
</ul>
<blockquote>
<p>注意：</p>
<p>当网络发现VLR标识IMSI附着的MS在一段时间内，没有与网络进行任何联系，将更改为该MS标识为隐含关机状态。</p>
</blockquote>
<p><strong>周期性位置更新的目的：</strong></p>
<ul>
<li><p>周期性的通知网络MS的可用性。</p>
</li>
<li><p>迫使MS在经过一定时间后，自动向网络报告它目前的位置，这样网络就可以随时了解MS的当前状态。</p>
</li>
</ul>
<p><strong>周期性位置更新需要特别注意的几点说明：</strong></p>
<ul>
<li><p>当T3212逾时后，MS启动周期性位置更新，进入位置更新程序。并将T3212清零，从新计时。</p>
</li>
<li><p>当T3212逾时时，MS处于无可用小区、有限服务、搜索PLMN的状态时，MS将延时启动位置更新，直到脱离这些状态。</p>
</li>
<li><p>当MS处于无可用小区、有限服务、搜索PLMN的状态时，T3212的值当保持原值不能改变。</p>
</li>
</ul>
<h2 id="IMSI附着"><a href="#IMSI附着" class="headerlink" title="IMSI附着"></a>IMSI附着</h2><pre><code>MS开机时，MS将把自己的开机状态通知给网络，向网络发送IMSI附着的报文。网络收到该消息后将注明用户状态，以便当寻呼发生时，可以发起寻呼。

IMSI的附着与分离就是在MSC/VLR中用户记录上附加一个二进制标志。

IMSI附着：标志为允许接入

IMSI分离：标志为不可接入



当用户开机时发现SIM卡中LAI与网络LAI号一致，则进行IMSI附着，其过程与VLR内部位置更新过程基本一样，不同的是，仅在位置更新请求，标注为IMSI附着。
</code></pre><blockquote>
<p>注：IMSI附着、分离是系统的一个可选项。</p>
</blockquote>
<h1 id="名词解释"><a href="#名词解释" class="headerlink" title="名词解释"></a>名词解释</h1><ul>
<li><p>IMSI:国际移动用户识别码（International Mobile Subscriber Identification Number）是区别移动用户的标志，储存在SIM卡中，可用于区别移动用户的有效信息</p>
</li>
<li><p>IMEI:国际移动设备识别码（International Mobile Equipment Identity）用于在GSM移动网络中识别每一部独立的手机，相当于手机的身份证号码，另CDMA手机采用MEID码，与IMEI码有所区别，但也是唯一的识别码</p>
</li>
<li><p>MS:移动台（mobile station）移动用户的终端设备，可以分为车载型、便携型和手持型。其中手持型俗称“手机”</p>
</li>
<li><p>LAC:位置区码（location area code）为寻呼而设置的一个区域，覆盖一片地理区域，初期一般按行政区域划分（一个县或一个区）,现在按寻呼量划分</p>
</li>
<li><p>LAI：位置区识别码（Location Area Identity）</p>
<p>用于移动用户的位置更新。其结构如下：</p>
<p>LAI=MCC+MNC+LAC</p>
<p>MCC=移动国家号，与IMSI中的MCC一样具有3个数字，用于识别一个国家，中国为460。</p>
<p>MNC=移动网号，识别国内GSM网，与IMSI中的MNC的值是一样的：</p>
<ul>
<li><p>00：中国移动</p>
</li>
<li><p>01：中国联通</p>
</li>
<li><p>02：中国移动TD</p>
</li>
<li><p>03：中国电信</p>
</li>
<li>11：中国电信4G</li>
</ul>
<p>LAC=位置区码，识别一个GSM网中的位置区</p>
</li>
<li><p>TAC：跟踪区域码（Tracking Area Code）在LTE网络中本参数定义了小区所属的跟踪区域码，一个跟踪区域可以涵盖一个或者多个小区</p>
</li>
<li><p>TAI：LTE的跟踪区标识（Tracking Area Identity），是由PLMN和TAC组成。</p>
<p>TAI = PLMN + TAC</p>
</li>
<li><p>PLMN：公共陆地移动网络（Public Land Mobile Network）</p>
<p>PLMN = MCC + MNC</p>
</li>
<li><p>VLR：拜访位置寄存器（Visitor Location Register），是一个动态数据库，一般与MSC合设，存储所管辖区域中MS的来话、去话呼叫所需检索的信息以及用户签约业务和附加业务的信息，例如客户的号码，所处位置区域的识别，向客户提供的服务等参数</p>
</li>
<li><p>HLR：归属位置寄存器（Home Location Register），是一个负责移动用户管理的数据库，永久存储和记录所辖区域内用户的签约数据，并动态地更新用户的位置信息，以便在呼叫业务中提供被呼叫用户的网络路由</p>
</li>
<li><p>MSC：移动交换中心（Mobile Switching Center），通信系统的核心网元之一，是在电话和数据系统之间提供呼叫转换服务和呼叫控制的设备。</p>
</li>
<li><p>TMSI：临时移动用户标识（Temporary Mobile Subscriber Identity），用户移动到某个位置区下时，由VLR分配的临时标识，只在一个位置区的某一段时间内有效</p>
</li>
<li><p>PVLR：位置更新前MS所属的VLR</p>
</li>
<li><p>T3212：周期性位置更新计时器，当T3212超时，将进行周期性位置更新</p>
</li>
<li><p>ANR：邻区自动优化（Automatic Neighbor Relatior）UE检查到信号更强的PCI时，UE上报MR消息（仅包含PCI，不包含CGI），由于系统中没有目标切换小区的邻区，eNodeB要求UE上报目标邻区的CGI，手机再次上报MR消息（包含CGI），ANR将自动在邻区列表中添加漏配邻区，这样用户就能连上漏配邻区</p>
</li>
<li><p>UE：用户终端（User Equipment），3G和4G网络中，用户终端就叫做UE，相当于2G网络中的MS</p>
</li>
<li><p>MR：测量报告（Measurement Report），包含当前小区基站广播信息实际接收值</p>
</li>
<li><p>PCI：物理小区标识（Physical Cell Identifier），LTE中终端以此区分不同小区的无线信号，LTE系统提供504个PCI，与扰码概念类似，不同小区间可复用，所以规划的时候需要合理分配，确保同频同PCI的小区下行信号之间不会互相产生干扰</p>
</li>
<li><p>CGI：全球小区识别码（Cell Global Identifier）是用来识别一个小区（基站/一个扇形小区）所覆盖的区域，CGI是在LAI的基础上再加小区识别码（CID）构成的。</p>
<p>其结构是</p>
<p>MCC+MNC+LAC+CID</p>
<p>在LTE网络中为MCC+MNC+ENODEB_ID+CID</p>
<p>MCC：移动国家码</p>
<p>MNC：移动网络码</p>
<p>LAC：位置区号码</p>
<p>CID：小区标识码</p>
<p>ENODEB_ID：eNodeB标识码</p>
<p>TAC：区域跟踪码</p>
<p>其中MCC\MNC\LAC为位置区标识（LAI）</p>
<p>其中CID为2Byte的BCD码，由各MSC自定</p>
</li>
<li><p>eNodeB：演进型Node B（Evolved Node B），简称eNB，LTE中基站的名称</p>
</li>
</ul>

      
    </div>
    <footer class="article-footer">
      <a data-url="http://yoursite.com/2018/08/21/PseudoBaseStation/" data-id="cjptgn1ls0004ev15utbk6n5s" class="article-share-link">Share</a>
      
      
    </footer>
  </div>
  
</article>


  
    <article id="post-finding-donors-1st" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2018/07/10/finding-donors-1st/" class="article-date">
  <time datetime="2018-07-10T08:12:48.000Z" itemprop="datePublished">2018-07-10</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2018/07/10/finding-donors-1st/">finding_donors_1st</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <p>项目介绍：<br>在这个项目中，你将使用1994年美国人口普查收集的数据，选用几个监督学习算法以准确地建模被调查者的收入。然后，你将根据初步结果从中选择出最佳的候选算法，并进一步优化该算法以最好地建模这些数据。你的目标是建立一个能够准确地预测被调查者年收入是否超过50000美元的模型。这种类型的任务会出现在那些依赖于捐款而存在的非营利性组织。了解人群的收入情况可以帮助一个非营利性的机构更好地了解他们要多大的捐赠，或是否他们应该接触这些人。虽然我们很难直接从公开的资源中推断出一个人的一般收入阶层，但是我们可以（也正是我们将要做的）从其他的一些公开的可获得的资源中获得一些特征从而推断出该值<br><br>这个项目的数据集来自<a href="https://archive.ics.uci.edu/ml/datasets/Census+Income" target="_blank" rel="noopener">UCI机器学习知识库</a>。这个数据集是由Ron Kohavi和Barry Becker在发表文章”Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid”之后捐赠的，你可以在Ron Kohavi提供的<a href="https://www.aaai.org/Papers/KDD/1996/KDD96-033.pdf" target="_blank" rel="noopener">在线版本</a>中找到这个文章。我们在这里探索的数据集相比于原有的数据集有一些小小的改变，比如说移除了特征’fnlwgt’ 以及一些遗失的或者是格式不正确的记录。</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"><span class="keyword">import</span> pandas <span class="keyword">as</span> pd</span><br><span class="line"><span class="keyword">import</span> scipy <span class="keyword">as</span> sp</span><br><span class="line"><span class="keyword">from</span> sklearn.metrics <span class="keyword">import</span> accuracy_score,precision_score,recall_score,f1_score</span><br><span class="line"><span class="keyword">from</span> sklearn.model_selection <span class="keyword">import</span> train_test_split</span><br><span class="line"><span class="keyword">from</span> sklearn.preprocessing <span class="keyword">import</span> MinMaxScaler</span><br><span class="line"><span class="keyword">from</span> sklearn.linear_model <span class="keyword">import</span> LogisticRegression</span><br><span class="line"><span class="keyword">from</span> sklearn <span class="keyword">import</span> metrics</span><br><span class="line"><span class="keyword">import</span> os</span><br><span class="line"><span class="keyword">from</span> sklearn.externals <span class="keyword">import</span> joblib</span><br><span class="line"><span class="keyword">import</span> matplotlib.pyplot <span class="keyword">as</span> plt</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">train_model</span><span class="params">(data)</span>:</span></span><br><span class="line">  x = data[data.columns[:<span class="number">-1</span>]]</span><br><span class="line">  y = data[data.columns[<span class="number">-1</span>]]</span><br><span class="line">  x_train,x_valid，y_train,y_valid = train_test_split(x,y,test_size=<span class="number">0.3</span>,random_state=<span class="number">0</span>)</span><br><span class="line">  model = LogisticRegression(penalty=<span class="string">'l2'</span>,C=<span class="number">10</span>,class_weight=<span class="string">'balanced'</span>)</span><br><span class="line">  model = fit(x,y)</span><br><span class="line">  <span class="comment">#joblib.dump(model,'lr_model.pkl')</span></span><br><span class="line">  y_valid_pred = model.predict(x_valid)</span><br><span class="line">  print(<span class="string">"valid datasets accuracy_score:"</span>,accuracy_score(y_valid,y_valid_pred))</span><br><span class="line">  print(<span class="string">"valid datasets precision_score:"</span>,precision_score(y_valid,y_valid_pred))</span><br><span class="line">  print(<span class="string">"valid datasets recall_score:"</span>,recall_score(y_valid,y_valid_pred))</span><br><span class="line">  print(<span class="string">"F1 values:"</span>,f1_score(y_valid,y_valid_pred))</span><br><span class="line">  y_valid_prob = model.predict_proba(x_valid)</span><br><span class="line">  fpr,tpr,thresholds = metrics.roc_curve(y_valid,y_vy_valid_prob)</span><br><span class="line">  auc = metrics.auc(fpr,tpr)</span><br><span class="line">  print(<span class="string">"auc"</span>,auc)</span><br><span class="line">  <span class="comment"># 输出ROC曲线</span></span><br><span class="line">  plt.figure(facecolor=<span class="string">'w'</span>)</span><br><span class="line">  plt.plot(fpr,tpr,c=<span class="string">'r'</span>,lw=<span class="number">2</span>,alpha=<span class="number">0.9</span>,label=<span class="string">'AUC%.3f'</span> % auc)</span><br><span class="line">  plt.plot((<span class="number">0</span>,<span class="number">1</span>),(<span class="number">0</span>,<span class="number">1</span>),c=‘b’,lw=<span class="number">1.5</span>,ls=<span class="string">'--'</span>)</span><br><span class="line">  plt.xlim((<span class="number">-0.01</span>,<span class="number">1.02</span>))</span><br><span class="line">  plt.ylim((<span class="number">-0.01</span>,<span class="number">1.02</span>))</span><br><span class="line">  plt.xticks(np.arange(<span class="number">0</span>,<span class="number">1.1</span>,<span class="number">0.1</span>))</span><br><span class="line">  plt.yticks(np.arange(<span class="number">0</span>,<span class="number">1.1</span>,<span class="number">0.1</span>))</span><br><span class="line">  plt.xlabel(<span class="string">'False Positive Rate'</span>,fontsize=<span class="number">14</span>)</span><br><span class="line">  plt.ylabel(<span class="string">'True Positive Rate'</span>,fontsize=<span class="number">14</span>)</span><br><span class="line">  plt.grid(<span class="keyword">True</span>)</span><br><span class="line">  plt.legend(loc=<span class="string">'lower right'</span>,fontsize=<span class="number">14</span>)</span><br><span class="line">  plt.title(<span class="string">"finding_donors ROC and AUC value"</span>,fontsize=<span class="number">17</span>)</span><br><span class="line">  plt.show()</span><br><span class="line">  <span class="keyword">return</span> model</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> __name__ == <span class="string">'__main__'</span>:</span><br><span class="line">  raw_data = pd.read_csv(<span class="string">"train.csv"</span>)</span><br><span class="line">  object_columns = raw_data.select_dtypes(include=[<span class="string">'object'</span>]).columns</span><br><span class="line">  num_columns = raw_data.select_dtypes(include=[np.number])</span><br><span class="line"></span><br><span class="line">  <span class="comment"># number MinMaxScaler</span></span><br><span class="line">  scaler = MinMaxScaler()</span><br><span class="line">  raw_data[num_columns] = scaler.fit_transform(raw_data[nnum_columns])</span><br><span class="line"></span><br><span class="line">  <span class="comment"># preprocessing categorical</span></span><br><span class="line">  <span class="keyword">for</span> name <span class="keyword">in</span> object_columns:</span><br><span class="line">    raw_data[name] = pd.categorical(raw_data[name]).codes</span><br><span class="line">  <span class="comment"># 训练数据输出预测结果</span></span><br><span class="line">  model = train_model(raw_data)</span><br><span class="line"></span><br><span class="line">  <span class="comment"># kaggle predict fingding donors   test datasets</span></span><br><span class="line"></span><br><span class="line">  test_data = pd.read_csv(<span class="string">'test.csv'</span>)</span><br><span class="line">  <span class="keyword">for</span> name <span class="keyword">in</span> test_data.columns:</span><br><span class="line">    test_data[<span class="string">'name'</span>] = pd.categorical(test_data[name]).codes</span><br><span class="line">  y_test_pred = model.predict(test_data)</span><br></pre></td></tr></table></figure>

      
    </div>
    <footer class="article-footer">
      <a data-url="http://yoursite.com/2018/07/10/finding-donors-1st/" data-id="cjptgn1m7000bev15nd6usbav" class="article-share-link">Share</a>
      
      
    </footer>
  </div>
  
</article>


  
    <article id="post-finding-donors-2nd" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2018/07/10/finding-donors-2nd/" class="article-date">
  <time datetime="2018-07-10T06:53:42.000Z" itemprop="datePublished">2018-07-10</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2018/07/10/finding-donors-2nd/">finding_donors_2nd</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <h1 id="机器学习项目"><a href="#机器学习项目" class="headerlink" title="机器学习项目"></a>机器学习项目</h1><p>为CharityML寻找捐献者</p>
<h2 id="监督学习"><a href="#监督学习" class="headerlink" title="监督学习"></a>监督学习</h2><p>项目介绍：<br>在这个项目中，你将使用1994年美国人口普查收集的数据，选用几个监督学习算法以准确地建模被调查者的收入。然后，你将根据初步结果从中选择出最佳的候选算法，并进一步优化该算法以最好地建模这些数据。你的目标是建立一个能够准确地预测被调查者年收入是否超过50000美元的模型。这种类型的任务会出现在那些依赖于捐款而存在的非营利性组织。了解人群的收入情况可以帮助一个非营利性的机构更好地了解他们要多大的捐赠，或是否他们应该接触这些人。虽然我们很难直接从公开的资源中推断出一个人的一般收入阶层，但是我们可以（也正是我们将要做的）从其他的一些公开的可获得的资源中获得一些特征从而推断出该值<br><br>这个项目的数据集来自<a href="https://archive.ics.uci.edu/ml/datasets/Census+Income" target="_blank" rel="noopener">UCI机器学习知识库</a>。这个数据集是由Ron Kohavi和Barry Becker在发表文章”Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid”之后捐赠的，你可以在Ron Kohavi提供的<a href="https://www.aaai.org/Papers/KDD/1996/KDD96-033.pdf" target="_blank" rel="noopener">在线版本</a>中找到这个文章。我们在这里探索的数据集相比于原有的数据集有一些小小的改变，比如说移除了特征’fnlwgt’ 以及一些遗失的或者是格式不正确的记录。</p>
<h2 id="可视化处理脚步"><a href="#可视化处理脚步" class="headerlink" title="可视化处理脚步"></a>可视化处理脚步</h2><p>see the bottom python script</p>
<h2 id="探索数据"><a href="#探索数据" class="headerlink" title="探索数据"></a>探索数据</h2><p>运行下面的代码单元以载入需要的Python库并导入人口普查数据。注意数据集的最后一列’income’将是我们需要预测的列（表示被调查者的年收入会大于或者是最多50,000美元），人口普查数据中的每一列都将是关于被调查者的特征。<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 检查你的Python版本</span></span><br><span class="line"><span class="keyword">from</span> sys <span class="keyword">import</span> version_info</span><br><span class="line"><span class="keyword">if</span> version_info.major != <span class="number">3</span> <span class="keyword">and</span> version_info.minor != <span class="number">5</span>:</span><br><span class="line">    <span class="keyword">raise</span> Exception(<span class="string">'请使用Python 3.5来完成此项目'</span>)</span><br></pre></td></tr></table></figure></p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 为这个项目导入需要的库</span></span><br><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"><span class="keyword">import</span> pandas <span class="keyword">as</span> pd</span><br><span class="line"><span class="keyword">from</span> time <span class="keyword">import</span> time</span><br><span class="line"><span class="keyword">from</span> IPython.display <span class="keyword">import</span> display <span class="comment"># 允许为DataFrame使用display()</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 导入附加的可视化代码visuals.py</span></span><br><span class="line"><span class="keyword">import</span> visuals <span class="keyword">as</span> vs</span><br><span class="line"></span><br><span class="line"><span class="comment"># 为notebook提供更加漂亮的可视化</span></span><br><span class="line">%matplotlib inline</span><br><span class="line"></span><br><span class="line"><span class="comment"># 导入人口普查数据</span></span><br><span class="line">data = pd.read_csv(<span class="string">"census.csv"</span>)</span><br></pre></td></tr></table></figure>
<h3 id="数据探索"><a href="#数据探索" class="headerlink" title="数据探索"></a>数据探索</h3><p> 首先我们对数据集进行一个粗略的探索，我们将看看每个类别里会有多少被调查，并且告诉我们大于50000美金一年收入的比例。</p>
<ul>
<li>总的记录数据，<code>n_records</code></li>
<li>年收入大于50，000美金的人数 <code>n_greater_50k</code></li>
<li>年收入小于等于50，000美金人数,<code>n_at_most_50k</code></li>
<li>年收入大于50，000美金人数的占比，<code>greater_percent</code></li>
</ul>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 总的记录数</span></span><br><span class="line">n_records = len(data)</span><br><span class="line">n_greater_50k = len(data[data[<span class="string">'income'</span>]==<span class="string">'&gt;50k'</span>])</span><br><span class="line">n_at_most_50k = len(data[data[<span class="string">'income'</span>]==<span class="string">'&lt;=50k'</span>])</span><br><span class="line">greater_percent = n_greater_50k/n_records</span><br><span class="line">print(<span class="string">"n_greater_50k values Rate:"</span>,greater_percent)</span><br></pre></td></tr></table></figure>
<h2 id="准备数据"><a href="#准备数据" class="headerlink" title="准备数据"></a>准备数据</h2><p>在数据能够被作为输入给机器学习算法之前，需要对原始数据进行清洗，格式化和重新组织，通常这些操作被称为预处理。本次数据不存在缺失值，所以没有必要处理无效和丢失值。</p>
<h3 id="获得特征和标签"><a href="#获得特征和标签" class="headerlink" title="获得特征和标签"></a>获得特征和标签</h3><p>income 列是我们需要的标签，记录一个人的年收入是否高于50k。因此需要剔除出来。<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># split datasets  features and label</span></span><br><span class="line">income_raw = data[<span class="string">'income'</span>]</span><br><span class="line">features_raw = data.drop(<span class="string">'income'</span>,axis=<span class="number">1</span>)</span><br></pre></td></tr></table></figure></p>
<p>###<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">vd.distribution(features_raw)</span><br></pre></td></tr></table></figure></p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">skewed = [<span class="string">'capital-gain'</span>,<span class="string">'capital-loss'</span>]</span><br><span class="line">features_raw[skewed] = data[skewed].apply(<span class="keyword">lambda</span> x: np.log(x+<span class="number">1</span>))</span><br><span class="line">vs.distribution(features_raw,ttransformed = <span class="keyword">True</span>)</span><br></pre></td></tr></table></figure>
<h3 id="Normalized-numberical-value-features"><a href="#Normalized-numberical-value-features" class="headerlink" title="Normalized numberical value features"></a>Normalized numberical value features</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">from</span> sklearn.preprocessing  <span class="keyword">import</span> MinMaxScaler</span><br><span class="line"><span class="comment"># initialization scaler</span></span><br><span class="line">scaler = MinMaxScaler()</span><br><span class="line">numberical = [<span class="string">'age'</span>,<span class="string">'education-num'</span>,<span class="string">'capital-gain'</span>,<span class="string">'capital-loss'</span>,<span class="string">'hours-per-week'</span>]</span><br><span class="line">features_raw[numberical] = scaler.fit_transform(data[numberical])</span><br><span class="line"><span class="comment"># category change to one hot encodeing</span></span><br><span class="line"><span class="keyword">from</span> sklearn <span class="keyword">import</span> preprocessing</span><br><span class="line">income = pd.Series(preprocessing.LabelEncoder().fit_transform(income_raw))</span><br><span class="line">features = pd.get_dummies(features_raw)</span><br></pre></td></tr></table></figure>
<h3 id="混洗和切分数据"><a href="#混洗和切分数据" class="headerlink" title="混洗和切分数据"></a>混洗和切分数据</h3><p>现在所有的 <em>类别变量</em> 已被转换成数值特征，而且所有的数值特征已被规一化。和我们一般情况下做的一样，我们现在将数据（包括特征和它们的标签）切分成训练和测试集。其中80%的数据将用于训练和20%的数据用于测试。然后再进一步把训练数据分为训练集和验证集，用来选择和优化模型。<br>用了stratify参数，training集和testing集的类的比例是 A：B= 4：1，等同于split前的比例（80：20）。通常在这种类分布不平衡的情况下会用到stratify。</p>
<p>将stratify=X就是按照X中的比例分配</p>
<p>将stratify=y就是按照y中的比例分配<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">from</span> sklearn <span class="keyword">import</span> train_test_split</span><br><span class="line">X_train,X_test,y_train,y_test = train_test_split(features,income,test_size=<span class="number">0.2</span>,random_state=<span class="number">0</span>,stratify=income)</span><br><span class="line">X_train,X_val,y_train,y_val = train_test_split(X_train,y_train,test_size=<span class="number">0.2</span>,random_state=<span class="number">0</span>,stratify=y_train)</span><br><span class="line"><span class="comment"># 显示切分的结果</span></span><br><span class="line"><span class="keyword">print</span> (<span class="string">"Training set has &#123;&#125; samples."</span>.format(X_train.shape[<span class="number">0</span>]))</span><br><span class="line"><span class="keyword">print</span> (<span class="string">"Validation set has &#123;&#125; samples."</span>.format(X_val.shape[<span class="number">0</span>]))</span><br><span class="line"><span class="keyword">print</span> (<span class="string">"Testing set has &#123;&#125; samples."</span>.format(X_test.shape[<span class="number">0</span>]))</span><br></pre></td></tr></table></figure></p>
<h2 id="评价模型性能"><a href="#评价模型性能" class="headerlink" title="评价模型性能"></a>评价模型性能</h2><p>在这一部分中，我们将尝试四种不同的算法，并确定哪一个能够最好地建模数据。四种算法包含一个天真的预测器 和三个你选择的监督学习器。</p>
<h3 id="评价方法和朴素的预测器"><a href="#评价方法和朴素的预测器" class="headerlink" title="评价方法和朴素的预测器"></a>评价方法和朴素的预测器</h3><p><em>CharityML</em>通过他们的研究人员知道被调查者的年收入大于\$50,000最有可能向他们捐款。因为这个原因<em>CharityML</em>对于准确预测谁能够获得\$50,000以上收入尤其有兴趣。这样看起来使用<strong>准确率</strong>作为评价模型的标准是合适的。另外，把<em>没有</em>收入大于\$50,000的人识别成年收入大于\$50,000对于<em>CharityML</em>来说是有害的，因为他想要找到的是有意愿捐款的用户。这样，我们期望的模型具有准确预测那些能够年收入大于\$50,000的能力比模型去<strong>查全</strong>这些被调查者<em>更重要</em>。我们能够使用<strong>F-beta score</strong>作为评价指标，这样能够同时考虑查准率和查全率：<br>$$F_{\beta}=(1+\beta^2)\cdot\frac{precision\cdot recall}{\beta^2 \cdot precision + recall}$$<br>尤其是，当 $\beta = 0.5$ 的时候更多的强调查准率，这叫做<strong>$F_{0.5}$ score</strong> （或者为了简单叫做F-score）。</p>
<h3 id="朴素预测器的性能"><a href="#朴素预测器的性能" class="headerlink" title="朴素预测器的性能"></a>朴素预测器的性能</h3><p>通过初步探索收入超过和不超过50，000美金的人数，可以看出超过50，000美金的人数并不多。如果预测所有人全部都超过50，000美金，那么准确率也会超过50%。这种预测器被称为朴素预测器，这样的一个预测器是评判一个模型表现的基线。</p>
<p>在不使用scikit-learn,需要自己根据公式计算。<br>true positive （TP：正类判定为正类）判定正确<br>false positive（FP：负类判定为正类）<br>false negative（FN：正类判定为负类）<br>True negative （TN：负类判定为负类）判定正确<br>错误率 error rate = （FP+FN）/（TN+TP+FN+FP）<br>准确率 accuracy = （TP+TN）/（TN+TP+FN+FP）<br>精准率 precision = TP/（TP+FP）<br>召回率 recall = TP/（TP+FN）<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">TP = float(len(y_val[y_val==<span class="number">1</span>]))</span><br><span class="line">FP = float(len(y_val[y_val==<span class="number">0</span>]))</span><br><span class="line">FN = <span class="number">0</span></span><br><span class="line">TN = <span class="number">0</span></span><br><span class="line">accuracy = (TP+TN)/(TP+FP+FN+TN)</span><br><span class="line">precision = TP/(TP+FP)</span><br><span class="line">recall = TP/(TP+FN)</span><br><span class="line">f_score = (<span class="number">1</span>+<span class="number">0.5</span>**<span class="number">2</span>)*precision*recall/((<span class="number">0.5</span>**<span class="number">2</span>)*precision+recall)</span><br></pre></td></tr></table></figure></p>
<p>当$\beta = 0.5$ 的时候更多的强调查准率，这叫做$F_{0.5 score}$或者简称为$F{-score}$。</p>
<h1 id="监督学习模型"><a href="#监督学习模型" class="headerlink" title="监督学习模型"></a>监督学习模型</h1><h3 id="模型1-支持向量机（SVM）"><a href="#模型1-支持向量机（SVM）" class="headerlink" title="模型1 支持向量机（SVM）"></a>模型1 支持向量机（SVM）</h3><p>选用kernel为了linear和rbf.现实中SVM广泛应用于视频和图像中人类行为的识别，根据图像判断人在干什么。<br>参考文献：<a href="http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=1334462" target="_blank" rel="noopener">http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=1334462</a><br>模型的优缺点：<br>优点：</p>
<ul>
<li>在非线性可分问题上表现优秀。</li>
<li>在训练数据量较小的表现良好。</li>
<li>抗数据的攻击能力好。<br>缺点：</li>
<li>在数据量较大的情况下，训练时间长导致难以实施。</li>
<li>解决多分类问题存在一定的困难。</li>
</ul>
<p>结合当前的finding donors 数据集的特点分析如下：</p>
<ul>
<li>首先，当前数据集的不是很大，属于小数据集。</li>
<li>其次，数据是否是线性的还未知。</li>
<li>最后预测目标是二分类问题，不是多分类，所以适合SVM<h3 id="K近邻"><a href="#K近邻" class="headerlink" title="K近邻"></a>K近邻</h3>使用K近邻法估计和绘制森林密度，体积和覆盖类型<br>参考文献：<a href="http://www.sciencedirect.com/science/article/pii/S0034425701002097" target="_blank" rel="noopener">http://www.sciencedirect.com/science/article/pii/S0034425701002097</a><br>模型的优缺点：<br>优点：</li>
<li>思想简单，容易理解和解释。</li>
<li>聚类效果不错。<br>缺点：</li>
<li>对异常值敏感。</li>
<li>需要提前确定k值。</li>
<li>局部最优解不是全局最优解（和初值的选取有关）。</li>
<li>算法复杂度不易控制O(NKm),迭代次数可能会很大，（考虑到m的值可能会很大）。<br>结合当前的finding donors 数据集的特点分析如下：</li>
<li>我们的数据可能存在异常值，但是可以剔除，降低对异常值的敏感程度。</li>
<li>数据量不大，所以计算时间不会太长。</li>
<li>因为是二分类，所以k值可以确定。<h3 id="集成方法（选择Random-Forest）"><a href="#集成方法（选择Random-Forest）" class="headerlink" title="集成方法（选择Random Forest）"></a>集成方法（选择Random Forest）</h3>应用场景：</li>
<li>随机森林应用于土地覆盖率</li>
<li>参考文献： <a href="http://www.sciencedirect.com/science/article/pii/S0924271611001304" target="_blank" rel="noopener">http://www.sciencedirect.com/science/article/pii/S0924271611001304</a><br>模型的优缺点：<br>优点：</li>
<li>准确率高。</li>
<li>能够有效的运行在大数据中。</li>
<li>能处理高维特征的输入样本，不需要降维。</li>
<li>能够评估各个特征在分类问题上的重要性。<br>缺点：</li>
<li>随机森林在一些噪音较大的分类和回归问题上会过拟合。</li>
<li>对于有不同取值的属性的数据，取值划分较多的属性会对随机森林产生更大的影响，所以随机森林在这种数据上产生的的属性权值是不可信的。<br>结合当前数据集，综合评价：</li>
<li>数据维度较高，处理起来很合适。</li>
<li>能够评估各个特征的重要性，可用作为一个很好的特征参考。</li>
<li>准确率较高，小范围的噪音不会过拟合。<h2 id="可视化脚步"><a href="#可视化脚步" class="headerlink" title="可视化脚步"></a>可视化脚步</h2><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">###########################################</span></span><br><span class="line"><span class="comment"># Suppress matplotlib user warnings</span></span><br><span class="line"><span class="comment"># Necessary for newer version of matplotlib</span></span><br><span class="line"><span class="keyword">import</span> warnings</span><br><span class="line">warnings.filterwarnings(<span class="string">"ignore"</span>, category = UserWarning, module = <span class="string">"matplotlib"</span>)</span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># Display inline matplotlib plots with IPython</span></span><br><span class="line"><span class="keyword">from</span> IPython <span class="keyword">import</span> get_ipython</span><br><span class="line">get_ipython().run_line_magic(<span class="string">'matplotlib'</span>, <span class="string">'inline'</span>)</span><br><span class="line"><span class="comment">###########################################</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> matplotlib.pyplot <span class="keyword">as</span> pl</span><br><span class="line"><span class="keyword">import</span> matplotlib.patches <span class="keyword">as</span> mpatches</span><br><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"><span class="keyword">import</span> pandas <span class="keyword">as</span> pd</span><br><span class="line"><span class="keyword">from</span> time <span class="keyword">import</span> time</span><br><span class="line"><span class="keyword">from</span> sklearn.metrics <span class="keyword">import</span> f1_score, accuracy_score</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">distribution</span><span class="params">(data, transformed = False)</span>:</span></span><br><span class="line">    <span class="string">"""</span></span><br><span class="line"><span class="string">    Visualization code for displaying skewed distributions of features</span></span><br><span class="line"><span class="string">    """</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># Create figure</span></span><br><span class="line">    fig = pl.figure(figsize = (<span class="number">11</span>,<span class="number">5</span>));</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Skewed feature plotting</span></span><br><span class="line">    <span class="keyword">for</span> i, feature <span class="keyword">in</span> enumerate([<span class="string">'capital-gain'</span>,<span class="string">'capital-loss'</span>]):</span><br><span class="line">        ax = fig.add_subplot(<span class="number">1</span>, <span class="number">2</span>, i+<span class="number">1</span>)</span><br><span class="line">        ax.hist(data[feature], bins = <span class="number">25</span>, color = <span class="string">'#00A0A0'</span>)</span><br><span class="line">        ax.set_title(<span class="string">"'%s' Feature Distribution"</span>%(feature), fontsize = <span class="number">14</span>)</span><br><span class="line">        ax.set_xlabel(<span class="string">"Value"</span>)</span><br><span class="line">        ax.set_ylabel(<span class="string">"Number of Records"</span>)</span><br><span class="line">        ax.set_ylim((<span class="number">0</span>, <span class="number">2000</span>))</span><br><span class="line">        ax.set_yticks([<span class="number">0</span>, <span class="number">500</span>, <span class="number">1000</span>, <span class="number">1500</span>, <span class="number">2000</span>])</span><br><span class="line">        ax.set_yticklabels([<span class="number">0</span>, <span class="number">500</span>, <span class="number">1000</span>, <span class="number">1500</span>, <span class="string">"&gt;2000"</span>])</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Plot aesthetics</span></span><br><span class="line">    <span class="keyword">if</span> transformed:</span><br><span class="line">        fig.suptitle(<span class="string">"Log-transformed Distributions of Continuous Census Data Features"</span>, \</span><br><span class="line">            fontsize = <span class="number">16</span>, y = <span class="number">1.03</span>)</span><br><span class="line">    <span class="keyword">else</span>:</span><br><span class="line">        fig.suptitle(<span class="string">"Skewed Distributions of Continuous Census Data Features"</span>, \</span><br><span class="line">            fontsize = <span class="number">16</span>, y = <span class="number">1.03</span>)</span><br><span class="line"></span><br><span class="line">    fig.tight_layout()</span><br><span class="line">    fig.show()</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">evaluate</span><span class="params">(results, accuracy, f1)</span>:</span></span><br><span class="line">    <span class="string">"""</span></span><br><span class="line"><span class="string">    Visualization code to display results of various learners.</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string">    inputs:</span></span><br><span class="line"><span class="string">      - learners: a list of supervised learners</span></span><br><span class="line"><span class="string">      - stats: a list of dictionaries of the statistic results from 'train_predict()'</span></span><br><span class="line"><span class="string">      - accuracy: The score for the naive predictor</span></span><br><span class="line"><span class="string">      - f1: The score for the naive predictor</span></span><br><span class="line"><span class="string">    """</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># Create figure</span></span><br><span class="line">    fig, ax = pl.subplots(<span class="number">2</span>, <span class="number">3</span>, figsize = (<span class="number">11</span>,<span class="number">7</span>))</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Constants</span></span><br><span class="line">    bar_width = <span class="number">0.3</span></span><br><span class="line">    colors = [<span class="string">'#A00000'</span>,<span class="string">'#00A0A0'</span>,<span class="string">'#00A000'</span>]</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Super loop to plot four panels of data</span></span><br><span class="line">    <span class="keyword">for</span> k, learner <span class="keyword">in</span> enumerate(results.keys()):</span><br><span class="line">        <span class="keyword">for</span> j, metric <span class="keyword">in</span> enumerate([<span class="string">'train_time'</span>, <span class="string">'acc_train'</span>, <span class="string">'f_train'</span>, <span class="string">'pred_time'</span>, <span class="string">'acc_val'</span>, <span class="string">'f_val'</span>]):</span><br><span class="line">            <span class="keyword">for</span> i <span class="keyword">in</span> np.arange(<span class="number">3</span>):</span><br><span class="line"></span><br><span class="line">                <span class="comment"># Creative plot code</span></span><br><span class="line">                ax[j/<span class="number">3</span>, j%<span class="number">3</span>].bar(i+k*bar_width, results[learner][i][metric], width = bar_width, color = colors[k])</span><br><span class="line">                ax[j/<span class="number">3</span>, j%<span class="number">3</span>].set_xticks([<span class="number">0.45</span>, <span class="number">1.45</span>, <span class="number">2.45</span>])</span><br><span class="line">                ax[j/<span class="number">3</span>, j%<span class="number">3</span>].set_xticklabels([<span class="string">"1%"</span>, <span class="string">"10%"</span>, <span class="string">"100%"</span>])</span><br><span class="line">                ax[j/<span class="number">3</span>, j%<span class="number">3</span>].set_xlabel(<span class="string">"Training Set Size"</span>)</span><br><span class="line">                ax[j/<span class="number">3</span>, j%<span class="number">3</span>].set_xlim((<span class="number">-0.1</span>, <span class="number">3.0</span>))</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Add unique y-labels</span></span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">0</span>].set_ylabel(<span class="string">"Time (in seconds)"</span>)</span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">1</span>].set_ylabel(<span class="string">"Accuracy Score"</span>)</span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">2</span>].set_ylabel(<span class="string">"F-score"</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">0</span>].set_ylabel(<span class="string">"Time (in seconds)"</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">1</span>].set_ylabel(<span class="string">"Accuracy Score"</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">2</span>].set_ylabel(<span class="string">"F-score"</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Add titles</span></span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">0</span>].set_title(<span class="string">"Model Training"</span>)</span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">1</span>].set_title(<span class="string">"Accuracy Score on Training Subset"</span>)</span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">2</span>].set_title(<span class="string">"F-score on Training Subset"</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">0</span>].set_title(<span class="string">"Model Predicting"</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">1</span>].set_title(<span class="string">"Accuracy Score on Validation Set"</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">2</span>].set_title(<span class="string">"F-score on Validation Set"</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Add horizontal lines for naive predictors</span></span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">1</span>].axhline(y = accuracy, xmin = <span class="number">-0.1</span>, xmax = <span class="number">3.0</span>, linewidth = <span class="number">1</span>, color = <span class="string">'k'</span>, linestyle = <span class="string">'dashed'</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">1</span>].axhline(y = accuracy, xmin = <span class="number">-0.1</span>, xmax = <span class="number">3.0</span>, linewidth = <span class="number">1</span>, color = <span class="string">'k'</span>, linestyle = <span class="string">'dashed'</span>)</span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">2</span>].axhline(y = f1, xmin = <span class="number">-0.1</span>, xmax = <span class="number">3.0</span>, linewidth = <span class="number">1</span>, color = <span class="string">'k'</span>, linestyle = <span class="string">'dashed'</span>)</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">2</span>].axhline(y = f1, xmin = <span class="number">-0.1</span>, xmax = <span class="number">3.0</span>, linewidth = <span class="number">1</span>, color = <span class="string">'k'</span>, linestyle = <span class="string">'dashed'</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Set y-limits for score panels</span></span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">1</span>].set_ylim((<span class="number">0</span>, <span class="number">1</span>))</span><br><span class="line">    ax[<span class="number">0</span>, <span class="number">2</span>].set_ylim((<span class="number">0</span>, <span class="number">1</span>))</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">1</span>].set_ylim((<span class="number">0</span>, <span class="number">1</span>))</span><br><span class="line">    ax[<span class="number">1</span>, <span class="number">2</span>].set_ylim((<span class="number">0</span>, <span class="number">1</span>))</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Create patches for the legend</span></span><br><span class="line">    patches = []</span><br><span class="line">    <span class="keyword">for</span> i, learner <span class="keyword">in</span> enumerate(results.keys()):</span><br><span class="line">        patches.append(mpatches.Patch(color = colors[i], label = learner))</span><br><span class="line">    pl.legend(handles = patches, bbox_to_anchor = (<span class="number">-.80</span>, <span class="number">2.53</span>), \</span><br><span class="line">               loc = <span class="string">'upper center'</span>, borderaxespad = <span class="number">0.</span>, ncol = <span class="number">3</span>, fontsize = <span class="string">'x-large'</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Aesthetics</span></span><br><span class="line">    pl.suptitle(<span class="string">"Performance Metrics for Three Supervised Learning Models"</span>, fontsize = <span class="number">16</span>, y = <span class="number">1.10</span>)</span><br><span class="line">    pl.tight_layout()</span><br><span class="line">    pl.show()</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">feature_plot</span><span class="params">(importances, X_train, y_train)</span>:</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># Display the five most important features</span></span><br><span class="line">    indices = np.argsort(importances)[::<span class="number">-1</span>]</span><br><span class="line">    columns = X_train.columns.values[indices[:<span class="number">5</span>]]</span><br><span class="line">    values = importances[indices][:<span class="number">5</span>]</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Creat the plot</span></span><br><span class="line">    fig = pl.figure(figsize = (<span class="number">9</span>,<span class="number">5</span>))</span><br><span class="line">    pl.title(<span class="string">"Normalized Weights for First Five Most Predictive Features"</span>, fontsize = <span class="number">16</span>)</span><br><span class="line">    rects = pl.bar(np.arange(<span class="number">5</span>), values, width = <span class="number">0.6</span>, align=<span class="string">"center"</span>, color = <span class="string">'#00A000'</span>, \</span><br><span class="line">                label = <span class="string">"Feature Weight"</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># make bar chart higher to fit the text label</span></span><br><span class="line">    axes = pl.gca()</span><br><span class="line">    axes.set_ylim([<span class="number">0</span>, np.max(values) * <span class="number">1.1</span>])</span><br><span class="line"></span><br><span class="line">    <span class="comment"># add text label on each bar</span></span><br><span class="line">    delta = np.max(values) * <span class="number">0.02</span></span><br><span class="line"></span><br><span class="line">    <span class="keyword">for</span> rect <span class="keyword">in</span> rects:</span><br><span class="line">        height = rect.get_height()</span><br><span class="line">        pl.text(rect.get_x() + rect.get_width()/<span class="number">2.</span>,</span><br><span class="line">                height + delta,</span><br><span class="line">                <span class="string">'%.2f'</span> % height,</span><br><span class="line">                ha=<span class="string">'center'</span>,</span><br><span class="line">                va=<span class="string">'bottom'</span>)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># Detect if xlabels are too long</span></span><br><span class="line">    rotation = <span class="number">0</span></span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> columns:</span><br><span class="line">        <span class="keyword">if</span> len(i) &gt; <span class="number">20</span>:</span><br><span class="line">            rotation = <span class="number">10</span> <span class="comment"># If one is longer than 20 than rotate 10 degrees</span></span><br><span class="line">            <span class="keyword">break</span></span><br><span class="line">    pl.xticks(np.arange(<span class="number">5</span>), columns, rotation = rotation)</span><br><span class="line">    pl.xlim((<span class="number">-0.5</span>, <span class="number">4.5</span>))</span><br><span class="line">    pl.ylabel(<span class="string">"Weight"</span>, fontsize = <span class="number">12</span>)</span><br><span class="line">    pl.xlabel(<span class="string">"Feature"</span>, fontsize = <span class="number">12</span>)</span><br><span class="line"></span><br><span class="line">    pl.legend(loc = <span class="string">'upper center'</span>)</span><br><span class="line">    pl.tight_layout()</span><br><span class="line">    pl.show()</span><br></pre></td></tr></table></figure>
</li>
</ul>

      
    </div>
    <footer class="article-footer">
      <a data-url="http://yoursite.com/2018/07/10/finding-donors-2nd/" data-id="cjptgn1m40009ev15wd66zze6" class="article-share-link">Share</a>
      
      
    </footer>
  </div>
  
</article>


  
    <article id="post-change-themes" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2018/07/10/change-themes/" class="article-date">
  <time datetime="2018-07-10T06:29:25.000Z" itemprop="datePublished">2018-07-10</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2018/07/10/change-themes/">change_themes</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <p>参考文献：<br><a href="https://hexo.io/themes/" target="_blank" rel="noopener">hexo官方相关主题</a><br><a href="http://www.codeblocq.com/assets/projects/hexo-theme-alpha-dust/2016/10/03/Welcome-to-Alpha-Dust/" target="_blank" rel="noopener">hexo配置Alpha主题连接</a></p>

      
    </div>
    <footer class="article-footer">
      <a data-url="http://yoursite.com/2018/07/10/change-themes/" data-id="cjptgn1lv0005ev15q1cjb3zl" class="article-share-link">Share</a>
      
      
    </footer>
  </div>
  
</article>


  
    <article id="post-Hong-yi-Lee-course" class="article article-type-post" itemscope itemprop="blogPost">
  <div class="article-meta">
    <a href="/2018/07/04/Hong-yi-Lee-course/" class="article-date">
  <time datetime="2018-07-04T00:49:31.000Z" itemprop="datePublished">2018-07-04</time>
</a>
    
  </div>
  <div class="article-inner">
    
    
      <header class="article-header">
        
  
    <h1 itemprop="name">
      <a class="article-title" href="/2018/07/04/Hong-yi-Lee-course/">Hong-yi_Lee_course</a>
    </h1>
  

      </header>
    
    <div class="article-entry" itemprop="articleBody">
      
        <table>
<thead>
<tr>
<th style="text-align:center"><a href="http://speech.ee.ntu.edu.tw/~tlkagk/index.html" target="_blank" rel="noopener">Home</a></th>
<th style="text-align:center"><a href="http://speech.ee.ntu.edu.tw/~tlkagk/talk.html" target="_blank" rel="noopener">DL/ML Tutorial</a></th>
<th style="text-align:center"><a href="http://speech.ee.ntu.edu.tw/~tlkagk/rtalk.html" target="_blank" rel="noopener">Research_Talk</a></th>
<th style="text-align:center"><a href="http://speech.ee.ntu.edu.tw/~tlkagk/research.html" target="_blank" rel="noopener">Research</a></th>
<th style="text-align:center"><a href="http://speech.ee.ntu.edu.tw/~tlkagk/publication.html" target="_blank" rel="noopener">Publication</a></th>
<th style="text-align:center"><a href="http://speech.ee.ntu.edu.tw/~tlkagk/courses.html" target="_blank" rel="noopener">Course</a></th>
<th style="text-align:center"><a href="http://speech.ee.ntu.edu.tw/~tlkagk/honor.html" target="_blank" rel="noopener">Honor</a></th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align:center"></td>
</tr>
</tbody>
</table>
<ul>
<li><a href="http://speech.ee.ntu.edu.tw/~tlkagk/courses_MLDS18.html" target="_blank" rel="noopener">Machine Learning and having it deep and structured (2018,Spring)</a></li>
<li><a href="http://speech.ee.ntu.edu.tw/~tlkagk/courses_ML17_2.html" target="_blank" rel="noopener">Machine Learning (2017,Fall)</a></li>
<li><a href="http://speech.ee.ntu.edu.tw/~tlkagk/courses_MLDS17_2.html" target="_blank" rel="noopener">Machine Learning and having it deep and structured (2017,Fall)</a></li>
<li><a href="http://speech.ee.ntu.edu.tw/~tlkagk/courses_ML17.html" target="_blank" rel="noopener">Machine Learning (2017,Spring)</a></li>
<li><a href="http://speech.ee.ntu.edu.tw/~tlkagk/courses_MLDS17.html" target="_blank" rel="noopener">Machine Learning and having it deep and structured (2017,Spring)</a></li>
<li><a href="http://speech.ee.ntu.edu.tw/~tlkagk/courses_ML16.html" target="_blank" rel="noopener">Machine Learning (2016,Fall)</a></li>
<li><a href="http://speech.ee.ntu.edu.tw/~tlkagk/courses_LA16.html" target="_blank" rel="noopener">Linear Algebra (2016,Spring)</a></li>
<li><a href="http://speech.ee.ntu.edu.tw/~tlkagk/courses_MLSD15_2.html" target="_blank" rel="noopener">Machine Learning and having it deep and structured (2015,Fall)</a></li>
<li>Machine Learning and having it deep and structured (2015,Spring)</li>
<li><a href="http://speech.ee.ntu.edu.tw/~tlkagk/courses_circuit14.html" target="_blank" rel="noopener">Circuit (2014,Fall)</a></li>
</ul>
<h1 id="推荐查看内容"><a href="#推荐查看内容" class="headerlink" title="推荐查看内容"></a>推荐查看内容</h1><ul>
<li><a href="http://speech.ee.ntu.edu.tw/~tlkagk/courses_MLDS18.html" target="_blank" rel="noopener">Machine Learning and having it deep and structured (2018,Spring)</a></li>
<li><a href="http://speech.ee.ntu.edu.tw/~tlkagk/courses_ML17_2.html" target="_blank" rel="noopener">Machine Learning (2017,Fall)</a></li>
</ul>
<ol>
<li>第一个是关于最新的深度学习内容，第二个是机器学习内容。</li>
<li>建议从2017年Machine Learning (2017,Fall)的视频开始看。然后Machine Learning and having it deep and structured (2018,Spring)。毕竟是最新的，2018年的课程中主要将的是deep learning 方面的支持。<br><a href="http://speech.ee.ntu.edu.tw/~tlkagk/courses.html" target="_blank" rel="noopener">参考文献</a></li>
</ol>

      
    </div>
    <footer class="article-footer">
      <a data-url="http://yoursite.com/2018/07/04/Hong-yi-Lee-course/" data-id="cjptgn1kw0000ev15qk9ndk93" class="article-share-link">Share</a>
      
      
    </footer>
  </div>
  
</article>


  


  <nav id="page-nav">
    
    <span class="page-number current">1</span><a class="page-number" href="/page/2/">2</a><a class="extend next" rel="next" href="/page/2/">Next &raquo;</a>
  </nav>

</section>
        
          <aside id="sidebar">
  
    

  
    
  <div class="widget-wrap">
    <h3 class="widget-title">Tags</h3>
    <div class="widget">
      <ul class="tag-list"><li class="tag-list-item"><a class="tag-list-link" href="/tags/Machine-Learning/">Machine Learning</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/linear-regression/">linear regression</a></li></ul>
    </div>
  </div>


  
    
  <div class="widget-wrap">
    <h3 class="widget-title">Tag Cloud</h3>
    <div class="widget tagcloud">
      <a href="/tags/Machine-Learning/" style="font-size: 20px;">Machine Learning</a> <a href="/tags/linear-regression/" style="font-size: 10px;">linear regression</a>
    </div>
  </div>

  
    
  <div class="widget-wrap">
    <h3 class="widget-title">Archives</h3>
    <div class="widget">
      <ul class="archive-list"><li class="archive-list-item"><a class="archive-list-link" href="/archives/2018/12/">December 2018</a></li><li class="archive-list-item"><a class="archive-list-link" href="/archives/2018/08/">August 2018</a></li><li class="archive-list-item"><a class="archive-list-link" href="/archives/2018/07/">July 2018</a></li></ul>
    </div>
  </div>


  
    
  <div class="widget-wrap">
    <h3 class="widget-title">Recent Posts</h3>
    <div class="widget">
      <ul>
        
          <li>
            <a href="/2018/12/18/SGD/">SGD</a>
          </li>
        
          <li>
            <a href="/2018/12/18/gradient-descent/">gradient_descent</a>
          </li>
        
          <li>
            <a href="/2018/12/06/linear-regression-gluon/">linear-regression-gluon</a>
          </li>
        
          <li>
            <a href="/2018/12/06/linear-regression-sources-md/">linear-regression-sources.md</a>
          </li>
        
          <li>
            <a href="/2018/12/05/regression/">regression</a>
          </li>
        
      </ul>
    </div>
  </div>

  
</aside>
        
      </div>
      <footer id="footer">
  
  <div class="outer">
    <div id="footer-info" class="inner">
      &copy; 2018 alex Chen<br>
      Powered by <a href="http://hexo.io/" target="_blank">Hexo</a>
    </div>
  </div>
</footer>
    </div>
    <nav id="mobile-nav">
  
    <a href="/" class="mobile-nav-link">Home</a>
  
    <a href="/archives" class="mobile-nav-link">Archives</a>
  
</nav>
    

<script src="//ajax.googleapis.com/ajax/libs/jquery/2.0.3/jquery.min.js"></script>


  <link rel="stylesheet" href="/fancybox/jquery.fancybox.css">
  <script src="/fancybox/jquery.fancybox.pack.js"></script>


<script src="/js/script.js"></script>



  </div>
</body>
</html>